Content Discovery

WSTG-INFO-04, WSTG-CONF-05

---

Goal

Map every accessible path, file, parameter, and virtual host on the target. Applications expose far more than what appears in the UI - backup files, old API versions, admin panels, and configuration files are routinely accessible to anyone who looks for them.

---

Wordlist Selection

The quality of your content discovery is limited by the quality of your wordlist. Kali ships wordlists in /usr/share/wordlists/ and /usr/share/seclists/ (SecLists).

# List available SecLists discovery wordlists
ls /usr/share/seclists/Discovery/Web-Content/

# Install SecLists if not present
sudo apt install seclists

Wordlist selection guide:

Wordlist	Path	Use Case
common.txt	/usr/share/wordlists/dirb/common.txt	Fast, broad - good first pass
big.txt	/usr/share/wordlists/dirb/big.txt	Broader coverage
directory-list-2.3-medium.txt	SecLists	Standard directory brute force
directory-list-2.3-big.txt	SecLists	Deep directory brute force
raft-large-directories.txt	SecLists	Large, high-quality directory list
raft-large-files.txt	SecLists	File-focused discovery
api/api-endpoints.txt	SecLists	API path discovery
CMS/wordpress.fuzz.txt	SecLists	WordPress-specific
CMS/drupal.txt	SecLists	Drupal-specific

---

feroxbuster - Recursive Directory Discovery

feroxbuster is the recommended starting tool for directory discovery. It recursively brute-forces subdirectories, making it far more effective than single-level tools on complex applications.

# Standard recursive scan
feroxbuster -u http://target.com -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt

# Scan with file extension probing
feroxbuster -u http://target.com -w /usr/share/seclists/Discovery/Web-Content/raft-large-files.txt \
  -x php,html,txt,bak,old,zip,tar.gz,sql,json,xml,config,log

# Limit recursion depth (prevents runaway scans on large sites)
feroxbuster -u http://target.com -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt \
  -d 3

# Filter out 404s and common false positive response sizes
feroxbuster -u http://target.com -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt \
  --filter-status 404,403 --filter-size 1234

# HTTPS target
feroxbuster -u https://target.com -k \
  -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt

# Throttle request rate (evasion)
feroxbuster -u http://target.com --rate-limit 50 \
  -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt

tip

Start with raft-large-directories.txt for structure, then follow up with raft-large-files.txt against interesting directories you find. The two-pass approach avoids overwhelming the target while maximizing coverage.

---

gobuster - Targeted Mode-Based Discovery

gobuster is faster than feroxbuster for single-level scans and has dedicated modes for DNS and virtual host enumeration.

# Directory mode
gobuster dir -u http://target.com \
  -w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt \
  -x php,html,txt,bak

# Show only specific response codes
gobuster dir -u http://target.com \
  -w /usr/share/wordlists/dirb/common.txt \
  -s 200,301,302,403

# DNS subdomain enumeration
gobuster dns -d target.com \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt

# Virtual host enumeration (identify name-based virtual hosts on the same IP)
gobuster vhost -u http://target.com \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  --append-domain

# With cookies (authenticated discovery)
gobuster dir -u http://target.com -w /usr/share/wordlists/dirb/common.txt \
  -c "session=abc123; auth=xyz"

---

ffuf - Flexible Fuzzing

ffuf uses the FUZZ keyword as a placeholder anywhere in the request - URL, headers, body, cookies. This makes it the most flexible discovery tool.

# Directory discovery
ffuf -u http://target.com/FUZZ -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt

# File discovery with extensions
ffuf -u http://target.com/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/raft-large-files.txt \
  -e .php,.html,.bak,.old,.txt,.xml,.json,.config,.log,.zip

# Virtual host discovery
ffuf -u http://target.com -H "Host: FUZZ.target.com" \
  -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  -fs <BASELINE_RESPONSE_SIZE>

# GET parameter fuzzing - discover hidden parameters
ffuf -u "http://target.com/page?FUZZ=test" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -fs <BASELINE_RESPONSE_SIZE>

# POST parameter fuzzing
ffuf -u http://target.com/login -X POST \
  -d "FUZZ=test&password=test" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -H "Content-Type: application/x-www-form-urlencoded"

# Filter by response size to remove false positives
ffuf -u http://target.com/FUZZ \
  -w /usr/share/seclists/Discovery/Web-Content/raft-large-directories.txt \
  -fs 4242

# Rate limiting
ffuf -u http://target.com/FUZZ -w /usr/share/wordlists/dirb/common.txt -rate 50

tip

For vhost discovery, first request the base URL and record the response size (-fs value). Use that size to filter out the default "not found" response, leaving only valid virtual hosts.

---

High-Value Files to Hunt For

These files are consistently worth targeting. Adjust extensions based on the detected tech stack.

Configuration and credential files

# .env files (Laravel, Node.js, Docker) - contain DB credentials, API keys, secrets
/.env
/.env.local
/.env.production
/.env.backup

# Web server configuration
/.htaccess
/web.config
/nginx.conf
/apache2.conf

# Application config
/config.php
/config.ini
/database.yml
/settings.py
/application.properties
/appsettings.json

Backup and temp files

# Common backup extensions for any file you find
/index.php.bak
/index.php~
/index.php.old
/index.php.orig
/config.php.bak
/admin.php.bak

# Archive files left on the server
/backup.zip
/backup.tar.gz
/www.zip
/site_backup.zip
/db.sql
/database.sql
/dump.sql

Source code and version control

# Exposed .git directory - dump the full source code
/.git/
/.git/HEAD
/.git/config
/.git/COMMIT_EDITMSG

# SVN
/.svn/
/.svn/entries

# Mercurial
/.hg/

danger

An exposed .git directory means the full application source code is recoverable. Use git-dumper to pull the repository:

# Install if needed
pip3 install git-dumper

# Dump the repository
git-dumper http://target.com/.git/ /tmp/target-source/

# Then browse/grep the source for credentials, logic flaws, and hidden endpoints
grep -r "password\|secret\|key\|token" /tmp/target-source/

Admin and management interfaces

# Generic admin paths
/admin
/admin/
/administrator
/admin.php
/admin/login
/admin/index.php
/_admin
/backend
/manage
/management
/dashboard
/control
/controlpanel
/cpanel

# CMS-specific
/wp-admin          # WordPress
/wp-login.php      # WordPress login
/administrator     # Joomla
/user/login        # Drupal
/typo3             # TYPO3

# Server management
/phpmyadmin
/pma
/adminer.php
/adminer
/manager/html      # Tomcat manager

# API documentation
/api/docs
/api/swagger
/swagger
/swagger-ui.html
/swagger.json
/openapi.json
/api-docs
/redoc

---

JavaScript File Mining

JavaScript files frequently contain API endpoints, authentication tokens, and application logic that is never visible in the UI.

# Extract all JS file URLs from a page
curl -s http://target.com | grep -oP '(src|href)="[^"]*\.js[^"]*"' | grep -oP '"[^"]+"' | tr -d '"'

# Download and search a JS file for interesting strings
curl -s http://target.com/static/app.bundle.js | \
  grep -oP '["'"'"'][/][a-zA-Z0-9/_-]+["'"'"']' | sort -u

# Look for API endpoints, tokens, secrets
curl -s http://target.com/static/app.js | \
  grep -iE "(api|endpoint|token|secret|key|password|auth|admin|internal|/v[0-9])" | \
  head -50

Tools like LinkFinder (if available) automate endpoint extraction from JavaScript:

python3 linkfinder.py -i http://target.com/static/app.js -o cli

---

Parameter Discovery

Hidden or undocumented parameters are a frequent source of injection vulnerabilities and access control bypasses.

# Fuzz GET parameters on a known page
ffuf -u "http://target.com/profile?FUZZ=1" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -fs <BASELINE_SIZE>

# Fuzz with multiple wordlists for better coverage
ffuf -u "http://target.com/admin?FUZZ=1" \
  -w /usr/share/seclists/Discovery/Web-Content/burp-parameter-names.txt \
  -mc 200,301,302,403 -fs <BASELINE_SIZE>

Interesting parameters to look for:

• debug=true, test=1, dev=1 - enable debug mode
• id=, uid=, user_id= - IDOR vectors
• file=, path=, page=, include= - LFI vectors
• url=, redirect=, next=, return= - SSRF / open redirect vectors
• role=, admin=, is_admin= - privilege escalation
• callback=, jsonp= - JSONP injection